Empirical Comparison of Full-Waveform Lidar Algorithms: Range Extraction and Discrimination Performance

As third-party lidar software manufacturers are increasingly adding support for full-waveform data, a common question is which algorithm(s) to implement. To this end, a new approach is needed to compare and contrast various lidar waveform processing strategies from a practical, operational perspective. Quality and type of information output, processing speed, suitability for particular applications, robustness against poor parameter selection, and more subjective measures related to user experience are of interest. This paper describes a new empirical method of comparing range extraction and discrimination performance of different algorithms, based on a ranging-lab setup with multiple, adjustable screen targets, with precisely-measured separations. We present the results of comparing three different algorithms described in the scientific literature. The results show distinct differences and also indicate that there is no “one-size-fits-all” approach: the choice of a specific algorithm and adjustable parameter settings are highly application-dependent. Introduction Prior to 2004, the commercial, topographic lidar market was dedicated to discrete-return systems, which record only a few (n ! 5) individual ranges per transmitted pulse. In principle, each recorded return in a discrete-return system corresponds to an individual laser reflection (i.e., an echo from one particular reflecting surface, which could be ground, or some elevated feature, such as a tree, pole, building, etc.). Ranging in discrete-return systems is done through hardware subsystems, typically comprising a constant fraction discriminator and time-interval meter (Baltsavias, 1999; Parrish et al., 2005). By recording just a few individual ranges, discrete-return systems obviate the PHOTOGRAMMETRIC ENGINEER ING & REMOTE SENS ING Augu s t 2011 825 Empirical Comparison of Full-Waveform Lidar Algorithms: Range Extraction and Discrimination Performance Christopher E. Parrish, Inseong Jeong, Robert D. Nowak, and R. Brent Smith need for heavy data storage and processing requirements. An alternative to the discrete-return concept is full-waveform (FW) lidar. In FW systems, each backscattered laser pulse received by the system is digitized at a high sampling rate (e.g., 500 MHz to 1.5 GHz). This process generates digitized waveforms (amplitude versus time) that are stored for subsequent processing and analysis. In contrast to discretereturn lidar, FW systems enable software-based ranging through processing of the recorded waveforms. The FW concept is not new; in fact, it has been utilized since the 1970s in bathymetric lidar systems, such as the National Aeronautics and Space Administration (NASA) Airborne Oceanographic Lidar (AOL) (Hoge et al., 1980), and in large-footprint, experimental, vegetation-mapping systems, including NASA’s Scanning Lidar Imagery of Canopies by Echo Recovery (SLICER) (Means et al., 1999), and Laser Vegetation Imaging Sensor (LVIS) (Blair et al., 1999). However, commercially-available, small-footprint topographic lidar systems have only been available since around 2004, starting with the Riegl LMS-Q560 scanner and turnkey systems built upon it, and followed shortly by systems of several other lidar manufacturers (e.g., Hug et al., 2004; Lemmens, 2007). Mallet and Bretar (2009) provide a review of full-waveform lidar and description of current systems. Since the introduction of small-footprint FW lidar systems into the commercial market, many advantages have been described in the published literature. First, FW systems greatly improve the target resolution (also referred to as “vertical discrimination distance”), defined as the minimum separation of targets in the range direction, such that each can be individually resolved. Discrete-return systems tend to have poor target resolution, typically due to a sizable (up to !3.5 m) dead zone after each recorded return, resulting from the inherent limitations of the receiver electronics (Nayegandhi et al., 2006; Wagner et al., 2008). Using highsampling-rate digitizers and software-based ranging, FW systems enable great improvement in target resolution. Furthermore, it may be possible to perform ranging more accurately in software, and certainly more alternatives are available in the ranging technique (Wagner et al., 2004; Harding, 2009). As a related benefit, FW systems enable generation of much denser, detail-rich point clouds, which provide enhanced information about vertical structure Photogrammetric Engineering & Remote Sensing Vol. 77, No, 8, August 2011, pp. 825–838. 0099-1112/11/7708–0825/$3.00/0 © 2011 American Society for Photogrammetry and Remote Sensing Christopher E. Parrish is with the NOAA/NGS Remote Sensing Division, University of New Hampshire, Center for Coastal and Ocean Mapping, Jere A. Chase Ocean Engineering Lab, 24 Colovos Road, Durham, NH 03824 (chris. [email protected]). Inseong Jeong is with NOAA/NGS Remote Sensing Research (DST), 1315 East West Highway, Silver Spring, MD 20910. Robert D. Nowak is with the University of WisconsinMadison, Department of Electrical and Computer Engineering, 3627 Engineering Hall, 1415 Engineering Drive, Madison, WI 53706. R. Brent Smith is with Optech Incorporated, 300 Interchange Way, Vaughan, Ontario, Canada, L4K 5Z8. (Persson et al., 2005; Reitberger et al., 2006; Parrish and Nowak, 2009). In addition to (or, in some cases, instead of) range information, radiometric and other types of information related to surface characteristics can also be extracted from waveform data. The transition from discrete-return to FW in commercial, topographic lidar can also be described as a fundamental change in philosophy, in which fixed, hardware-based subsystems are replaced with customizable software alternatives. Sophisticated digital signal processing overcomes the need for sophisticated hardware, and limitless customization is enabled. All ranging strategies (whether implemented in hardware or software) involve engineering tradeoffs. The major difference is that a hardware-based solution remains fixed, once implemented by the lidar system manufacturer; in software, the user can select and/or tune the algorithm to best suit their application, be it floodplain mapping, forestry, building modeling, coastal marsh vegetation mapping, or airport obstruction surveying, for example. However, there is also an intrinsic tradeoff involved in the FW approach: in enabling more to be done in software, FW systems necessitate additional processing and algorithms to perform that processing. To date, several waveform processing strategies (and variations of the general approaches) have been described in the scientific literature (see, e.g., Hofton et al., 2000; Persson et al., 2005; Jutzi and Stilla, 2006; Chauve et al., 2007; Mallet and Bretar, 2009, and references therein). Unfortunately, third-party commercial software for processing and analyzing full-waveform data has lagged a bit behind both the new hardware development and the scientific community’s algorithm development. Thus, as many third-party software developers are currently at the stage of adding full-waveform processing capabilities, a common dilemma is which waveform processing algorithm(s) to implement. To make this determination, the software developers would like to know how the different algorithms compare, not just from a theoretical perspective, but in terms of real-world experience in implementing and using the different algorithms operationally. The objective of this paper is to help address these needs by: (a) presenting a new empirical approach to comparing lidar waveform processing algorithms in terms of range extraction and discrimination performance, and (b) using this experimental technique to compare three lidar waveform processing strategies described in the literature. Specifically, we present results of an experiment conducted using an Optech, Inc. topographic lidar system in a controlled ranging lab environment with multiple, adjustable targets. These results enable a robust comparison of the three algorithms, including advantages and disadvantages of each, and conclusions that may assist software providers in implementing FW support. Methods The three algorithms selected to test in this study are listed below. There are certainly different (or additional) algorithms that we could have selected, including, for example, the B-splines approach described in Roncat et al. (2010), the Average Square Difference Function (ASDF) method described in Wagner et al. (2007), the decomposition approach of Chauve et al. (2007) using refined peak detection and Lognormal or generalized Gaussian functions, or well-known signal processing techniques that could be adapted for lidar waveform processing, such as matched filtering (Turin, 1960). Ultimately, our selection was guided by three main criteria. First, we sought algorithms that were fundamentally different from one another, or broadly representative of different approaches to waveform processing. Second, we considered only algorithms for which source code was available or which were described in sufficient detail in the published literature that we could faithfully and efficiently reconstruct the authors’ steps in implementing them. Third, in order to keep the scope of the study manageable, we limited the total number of algorithms tested to three. The fact that the experimental design for comparing the different algorithms was new and needed to be tested and refined as a critical component of the study was a primary factor in limiting the number of algorithms tested. To enable fair comparisons of target resolution, the parameters of each algorithm were adjusted to maximize target discrimination while constraining the mean false alarm rate to !1% (see the Experiment Section for further details of the parameter tuning). As an aside, another form of comparison would be to produce receiver operating characteristic (ROC) curves for the different algorithms, but for our target resolution comparisons and application areas of interest, such as, airport obstruction surveying, we found it most insightful to constrain the false alarm rate to a specified value and examine how target discrimination falls off with decreasing target separation for the three algorithms. Gaussian Decomposition Gaussian decomposition is currently the most widelyapplied topographic lidar waveform processing strategy, based on numbers of known implementations and published papers (e.g., Hofton et al., 2000; Persson et al., 2005; Wagner et al., 2006; Reitberger et al., 2006). In Gaussian decomposition, each lidar waveform is modeled as a linear combination of Gaussian components (i.e., an N-component Gaussian mixture model):